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Overview 



Young people who drop out of high school face long odds of success in a labor market that in- 
creasingly values education and skills. This report presents interim results from a rigorous, ongoing 
evaluation of the National Guard Youth ChalleNGe Program, which aims to “reclaim the lives of at- 
risk youth” who have dropped out of high school. ChalleNGe is an intensive residential program that 
currently operates in more than half the states. More than 90,000 young people have completed the 
program since it was launched in the early 1990s. MDRC, a nonprofit, nonpartisan research organi- 
zation, is conducting the evaluation in collaboration with the Mac Arthur Foundation Research 
Network on Transitions to Adulthood. Several private foundations and the U.S. Department of 
Defense are funding the evaluation. 

The 17-month ChalleNGe program is divided into three phases: Pre-ChalleNGe, a demanding two- 
week orientation and assessment period; a 20-week Residential Phase built around eight core 
components designed to promote positive youth development; and a one-year Postresidential Phase 
featuring a structured mentoring program. During the first two phases, participants live at the 
program site, often on a military base. The environment is “quasi-military,” though there are no 
requirements for military service. 

The evaluation uses a random assignment design. Because there were more qualified applicants than 
slots, a lottery-like process was used to decide which applicants were admitted to the program. The 
young people who were admitted (the program group) are being compared over time with those who 
were not admitted (the control group); any significant differences that emerge between the groups 
can be attributed to ChalleNGe. About 3,000 young people entered the study in 10 ChalleNGe 
programs in 2005-2006. 



Interim Results 

A comprehensive survey was administered to about 1 ,200 young people in the program and control 

groups an average of 21 months after they entered the study. Key findings from the survey include: 

• The program group was much more likely than the control group to have obtained a high 
school diploma or a General Educational Development certificate (GED) and to have 
earned college credits. For example, about 61 percent of the program group had earned a dip- 
loma or a GED, compared with 36 percent of the control group. 

• At the time of the survey, program group members were somewhat more likely to be 
engaged in productive activities. For example, 72 percent of the program group were working, 
in school or training, or in the military, compared with 66 percent of the control group. 

• Young people in the two groups were equally likely to have been arrested in the year prior 
to the survey, but the program group was less likely to have been convicted of a crime or 
to have engaged in certain delinquent acts. 

• There were few differences between groups in measures of physical or mental health. 

Differences between groups that were measured at an earlier point in the study had disappeared 
by the 21 -month point. 

These interim results are impressive, but longer-term follow-up will be critical to understanding the 

full story of the program’s effects. Results from a 36-month survey should be available by late 2010. 

iii 




Preface 



High school dropout rates remain stubbornly high even as broad economic shifts con- 
tinue to erode opportunities for workers without postsecondary education. Finding ways to 
reengage high school dropouts and help them move forward in education and the labor market 
is a pressing national priority. The National Guard Youth ChalleNGe program has served more 
than 90,000 young high school dropouts since it was created in the early 1990s. The program, 
which now operates in more than half the states, uses an unusual model that combines an inten- 
sive residential phase with military-style discipline, a comprehensive focus on activities thought 
to promote positive youth development, and a postresidential mentoring program. 

The rigorous, random assignment evaluation of ChalleNGe described in this report is 
particularly timely, as the current administration has emphasized the importance of obtaining 
hard evidence on the effectiveness of federally funded programs. Early results from the evalua- 
tion, released in 2009, have already drawn attention to ChalleNGe, and Congress recently voted 
to increase the share of program costs that can be paid by the U.S. Department of Defense, a 
critical change for cash-strapped states. 

The evaluation represents a true collaboration. It is funded by a group of private founda- 
tions and the Department of Defense, and is led by MDRC, working closely with scholars from 
the MacArthur Foundation Research Network on Transitions to Adulthood. Twelve state Chal- 
leNGe programs agreed to participate, and 10 of them were eventually able to enroll about 
3,000 young people into the study sample. 

The interim results presented here are based on a survey administered to young people 
in the program and control groups an average of 21 months after they entered the study, when 
most were 18 or 19 years old. The program group members had completed the ChalleNGe pro- 
gram by that time. The survey found that young people who had access to ChalleNGe were 
much more likely than those in the control group to have obtained a high school diploma or a 
General Educational Development (GED) certificate. They were also somewhat more likely to 
be working, attending college, enlisted in the military, or engaging in other productive activities. 

While encouraging, these interim results are not the final word on ChalleNGe. Another 
survey, administered about 36 months after young people entered the study, is nearing comple- 
tion. It will be critical to see how the participants in the study have fared over a longer period, 
particularly because the survey was administered at a time when unemployment rates for disad- 
vantaged youth have reached crisis levels. Those results should be available by the end of 2010. 
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Executive Summary 



Young people who drop out of high school face long odds of success in a labor market 
that increasingly values education and skills. Nationally, about 30 percent of high school 
freshmen do not graduate in four years; in the 50 largest U.S. cities, the dropout rate is closer to 
50 percent. 

This report presents interim results from a rigorous, ongoing evaluation of the National 
Guard Youth ChalleNGe Program, which aims to “reclaim the lives of at-risk youth” who have 
dropped out of high school and give them the skills and values to succeed as adults. ChalleNGe 
is an intensive residential program that currently operates in more than half the states. More than 
90,000 young people have completed the program since it was launched in the early 1990s. 
MDRC, a nonprofit, nonpartisan research organization, is conducting the evaluation in collabo- 
ration with the MacArthur Foundation Research Network on Transitions to Adulthood. Several 
private foundations and the U.S. Department of Defense are funding the evaluation. 



The ChalleNGe Program 

The ChalleNGe approach grew out of a project by the Center for Strategic and Interna- 
tional Studies (CSIS) in the late 1980s and early 1990s that sought to develop new approaches 
for out-of-school youth. Staff in the National Guard Bureau in the U.S. Department of Defense 
developed the specific program model. They had concluded that many existing programs for 
disadvantaged youth were “focused on the symptomatic behaviors without understanding and 
addressing the underlying causes” and “placed limited, if any, focus on the post-program 
phase.” Thus, they designed ChalleNGe to be: 

...an intervention, rather than a remedial program. We would deal with the 
symptoms and underlying causes in a construct that fully embraced a “whole 
person” change and readied the students for the post-program environment. 

We would arm them with the skills and experiences necessary to succeed and 
we would ensure there was “a way back” to mainstream society. 1 

In 1993, Congress funded a 10-site pilot of ChalleNGe. Funding was made permanent 
in 1998, and today there are 32 ChalleNGe programs in 27 states and Puerto Rico. 

States operate ChalleNGe programs under a Master Cooperative Agreement with the 
National Guard Bureau. Most states operate a single “100-bed” ChalleNGe program, serving a 



’Donohue (2008). 



ES-1 




total of about 200 participants per year in two class cycles. A few states operate multiple 
programs or larger programs. The funding level for ChalleNGe — about $14,000 per participant 
— has not changed since the early 1990s. The federal government currently pays 60 percent of 
the cost of the state programs, and states pay the remaining 40 percent. 2 

Although there is considerable room to tailor the program model to local conditions, the 
basic structure of the ChalleNGe program is the same in all states. The program is open to 
young people between the ages of 16 and 18 who have dropped out of (or been expelled from) 
school, are unemployed, drug-free, and not heavily involved with the justice system. 3 The 
program is open to both males and females, though about 80 percent of the participants are 
male. There are no income-based eligibility criteria. 

The 17 -month program is divided into three phases: the Pre-ChalleNGe Phase (two 
weeks), the Residential Phase (20 weeks), and the Postresidential Phase (one year). During 
the first two phases (totaling 22 weeks), the participants live at the program site, often on a 
military base. 

The first phase, Pre-ChalleNGe, is a physically and psychologically demanding assess- 
ment and orientation period. Candidates are introduced to the program’s rules and expectations; 
learn military bearing, discipline, and teamwork; and begin physical fitness training. 

Candidates who complete Pre-ChalleNGe are formally enrolled in the program as “ca- 
dets” and move to the second phase. The curriculum for the 20-week Residential Phase is 
structured around eight core components that reflect current thinking about how to promote 
positive youth development: Leadership/Followership, Responsible Citizenship, Service to 
Community, Life-Coping Skills, Physical Fitness, Health and Hygiene, Job Skills, and Aca- 
demic Excellence. Cadets spend the largest share of each day in the education component. 
Most programs help participants prepare for the GED exam, but a few of them can offer a high 
school diploma. 

The program environment is described as “quasi-military”: The cadets are divided into 
platoons and squads, live in barracks, have their hair cut short, wear uniforms, and are subject 
to military-style discipline. The daily schedule is highly structured with almost no “down 
time,” and the cadets are closely supervised by staff at all times. While ChalleNGe uses 



2 The ChalleNGe legislation was amended in late 2009, raising the maximum federal share of program 
costs to 75 percent. 

3 In order to be eligible for ChalleNGe, candidates must be 16 to 18 years of age and enter the program 
before their nineteenth birthday; a high school dropout/expellee; a citizen or legal resident of the United States 
and a resident of the state in which the program is conducted; unemployed; not currently on parole or on 
probation for anything other than juvenile status offenses, not serving time or awaiting sentencing, not under 
indictment or charged, and not convicted of a felony or a capital offense; and drug-free. 
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military structure, discipline, facilities, and staff to accomplish its objectives, participation in 
the program is voluntary, and there are no requirements for military service during the program 
or afterward. 

Toward the end of the Residential Phase, the cadets work with staff to arrange a post- 
residential “placement.” Acceptable placements include employment, education, and military 
service. 



The cadets who successfully complete the Residential Phase move into the one -year 
Postresidential Phase, which involves a structured mentoring program. The ChalleNGe mentor- 
ing program is unusual, in that young people nominate their own mentors during the application 
process. ChalleNGe initiates the mentoring relationship partway through the Residential Phase, 
after the staff screen and train the mentors. The staff then maintain contact with both the 
program’s graduates and their mentors at least monthly during the Postresidential Phase to help 
solve problems and to monitor the youths’ progress. 



The ChalleNGe Evaluation 

The National Guard Bureau collects extensive data on program participation and partic- 
ipants’ outcomes. However, for some time, officials in the Department of Defense and many 
ChalleNGe program directors have been eager to obtain more rigorous data on what difference 
the program makes. The National Guard Bureau’s outcome data do not address this question 
because there is no way to know to what extent the outcomes that program participants or 
graduates achieve are actually attributable to their participation in ChalleNGe; the program 
serves relatively motivated young people who were determined to make a change and thus 
might make progress without ChalleNGe. Thus, in 2004, the officials and directors began 
working with MDRC and the Mac Arthur Foundation Research Network on Transitions to 
Adulthood to explore the possibility of conducting a random assignment evaluation of the 
program. Ultimately, the Department of Defense agreed to fund 20 percent of the evaluation, 
and MDRC raised the remaining 80 percent from private foundations. 

In 2005, 12 state ChalleNGe programs agreed to participate in the evaluation. These 
programs were not chosen randomly. Rather, there was an effort to identify programs that had 
stable staffing and that tended to receive more applicants than they could serve. 

The evaluation uses a random assignment research design in which a group of young 
people who applied to ChalleNGe and were invited to participate (the program group) is 
being compared over time with a second group (the control group) who applied to ChalleNGe 
and were deemed acceptable, but were not invited to participate. Random assignment was 
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conducted only during class cycles in which there were substantially more applicants than 
program slots. 

Because the study’s participants were assigned to the program group or the control 
group through a random process, one can be confident that any significant differences that 
emerge between the groups over time — for example, differences in educational attainment or 
employment rates — suggest that ChalleNGe was effective. These differences are described as 
impacts. 



Ultimately, random assignment was conducted for 18 class cycles across 10 programs 
in 2005 and 2006 (two of the sites that agreed to participate were unable to conduct random 
assignment because there were not substantially more qualified applicants than slots). About 
3,000 young people entered the study. 

A series of surveys is being administered to the program and control groups over time. 
The first survey, a very brief interview, was conducted about nine months after the young 
people entered the study — not long after participants in the program group had completed the 
residential phase of the program. The results from that survey, presented in an earlier report, 
were quite promising. 

This report presents the results from the second survey, a much more extensive inter- 
view conducted an average of 21 months after participants entered the study, after the postresi- 
dential phase of the program had ended. Just under 1,200 young people were interviewed, and 
the response rate was 79 percent. Most respondents were either 18 or 19 years old when they 
were interviewed. 



Results from the 21 -Month Survey 

• The program group was much more likely than the control group to 
have earned a GED and somewhat more likely to have obtained a high 
school diploma; the program group was also more likely to have earned 
college credits. 

As shown in the top panel of Table ES.l, about 61 percent of the program group re- 
ported on the survey that they had earned a high school diploma or a GED. The corresponding 
figure for the control group was 36 percent. The asterisks show that the difference between 
groups — about 24 percentage points — was statistically significant, meaning that it is very 
unlikely to have arisen by chance. As expected given the program model, many more program 
group members had earned a GED than a high school diploma, and the program’s impact was 
concentrated on GED receipt. Nevertheless, there was also a statistically significant increase in 
the percentage of sample members with a high school diploma. Program group members were 
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National Guard Youth ChalleNGe Program 

Table ES.l 

Impacts on Selected Outcomes from the 21-Month Survey 



Outcome 


Program 

Group 


Control 

Group 


Impact 


P- Value 31 


Educational attainment (%) 

Earned high school diploma or GED certificate 


60.5 


36.4 


24 i *** 


0.000 


High school diploma 


22.0 


16.3 


5.7 ** 


0.013 


GED certificate 


48.3 


21.9 


26.5 *** 


0.000 


Earned any college credit 


24.8 


9.6 


15.1 *** 


0.000 


Current status 

Currently in (%) 

High school or GED prep 


16.1 


26.0 


_9 9 *** 


0.000 


College courses 


11.6 


7.0 


4 ^ *** 


0.009 


Job training 


7.2 


6.5 


0.7 


0.648 


Currently working (%) 


55.0 


50.1 


4.9 * 


0.093 


Average weekly earnings ($) 


209 


169 


39 *** 


0.009 


Currently enlisted in the military (%) 


10.9 


6.2 


4 7 *** 


0.004 


Currently involved in any of the above activities ’ 3 (%) 


72.1 


66.4 


5.7 ** 


0.033 


Has high school diploma or GED certificate and is currently 

involved in any of the above activities ' 3 (%) 45.5 


23.1 


22.4 *** 


0.000 


Crime and delinauencv 

In the past 12 months 
Arrested (%) 


26.3 


25.4 


0.9 


0.719 


Convicted (%) 


8.9 


13.2 


-4.2 ** 


0.020 


Any self-reported delinquency (%) 


62.6 


66.5 


-3.9 


0.168 


Number of delinquent incidents 


1.6 


1.9 


-0.3 ** 


0.014 


Health (%) 

Overall health good or excellent 


71.0 


69.4 


1.6 


0.548 


Obese c 


9.8 


11.3 


-1.5 


0.425 


Serious psychological distress 11 


5.4 


5.1 


0.3 


0.844 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21-month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 
10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

“Standard errors are presented in this report for all impacts with a p-value of 0.000. Following are the 
standard errors for all impacts with a p-value of 0.000 (presented in the order in which they appear on the 
table): 2.827, 2.773, 2.264, 2.358,and 2.734. 

b This measure includes any employment, school or GED programs, vocational training, military 
activities, or any residential programs (not listed separately above). 

C A person is defined as obese if his or her Body Mass Index (BMI) is 30 or higher. 

d The K 6 scale is the sum of the responses to six questions asking how often a respondent experienced 
symptoms of psychological distress. The scale ranges from 0 to 24. A score of 13 points or more on the K 6 
scale is considered an indication of serious psychological distress. 
http://www.hcp.med.harvard.edu/ncs/k 6 _scales.php. 
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more than twice as likely to report that they had earned at least one college credit, though 
relatively few people in either group reported that they were currently in college when inter- 
viewed (second panel). 

• Young people in the program group were more likely to be engaged in 
productive activities. 

The second panel of Table ES.l describes sample members’ activities at the point that 
they responded to the survey. The table shows that control group members were more likely to 
be in high school or a GED preparation program, and that program group members were more 
likely to be in college, working, or in the military. 

Overall, about 72 percent of the program group was engaged in one of these productive 
activities, compared with 66 percent of the control group. A separate measure examined the 
percentage of each group that had completed a diploma or GED and was engaged in a produc- 
tive activity, a rough indicator of progress in the transition to adulthood. The program group 
was about twice as likely to be in this status. 

• There was no significant difference between groups in the percentage 
arrested in the year prior to the survey; the program group, however, 
was less likely to have been convicted of a crime and to have engaged in 
certain types of delinquency. 

As shown in Table ES.l, relatively large percentages of both groups reported involve- 
ment with the criminal justice system or delinquent activities. For example, about one-fourth of 
each group reported being arrested in the previous 12 months, and almost two-thirds of each 
group reported that they had engaged in at least one of 13 specific delinquent activities (for the 
most part, the delinquent activities were relatively minor, such as fighting). Although there were 
no significant differences between groups in either of these measures, the program group was 
less likely to report a conviction and also reported fewer delinquent activities. 

• There were no systematic differences between the groups on measures of 
current physical or mental health; most respondents in both groups re- 
ported being in relatively good health. 

At the nine-month point, the program group reported better health and less obesity. 
However, as shown in Table ES.l, those differences were no longer evident at the 21 -month 
point. There was also no difference between groups in a general measure of psychological 
distress. Other analyses, not shown in the table, found few differences in living arrangements or 
other measures of well-being. 
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Conclusions and Next Steps 

The results presented here constitute the second chapter in the unfolding story of the 
ChalleNGe program’s impacts. Twenty-one months after entering the study, at age 18 or 19, 
young people in the program group, who had access to ChalleNGe, were substantially more 
likely to have earned a high school diploma or a GED and were also more likely to be engaged 
in productive activities such as working or attending school. Those in the program group were 
also less likely to have been convicted of a crime and appeared to have engaged in few delin- 
quent activities. 

While quite promising, the 21 -month results also suggest some reasons for caution. As 
might be expected, many of the differences measured between the groups at 21 months were 
smaller than the corresponding differences at the nine-month point. Moreover, because many 
members of the control group were attending high school or GED preparation classes when 
they were interviewed, the program group’s advantage in high school completion may continue 
to narrow over time. Relatively few program group members were in college, so it is not yet 
clear how many ChalleNGe graduates will use their high school credentials to obtain further 
education or training that may qualify them for higher-paying jobs. 

Results from the 36-month survey, scheduled for late 2010, will fill in the next chapter 
in this important story. The next report will also expand on analyses that are presented in 
preliminary form in this report — for example, analysis of the program’ s impacts on measures 
of psychosocial development, and an examination of the program’ s impacts for different types 
of youth or different ChalleNGe programs. 
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About MDRC 



MDRC is a nonprofit, nonpartisan social policy research organization dedicated to learning 
what works to improve the well-being of low-income people. Through its research and the 
active communication of its findings, MDRC seeks to enhance the effectiveness of social and 
education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to learn not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy 
areas and target populations. Once known primarily for evaluations of state welfare-to-work 
programs, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Child Development 

• Improving Public Education 

• Promoting Successful Transitions to Adulthood 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 




